1. Field of the Invention
This invention generally relates to manipulating video content and more specifically to performing spatio-temporal segmentation of video content.
2. Description of the Related Art
The sharing of video content on websites has developed into a worldwide phenomenon, supported by dozens of websites. Many thousands of videos are posted every day, and this number is increasing as the tools and opportunities for capturing video become easier to use and more widespread. Millions of people watch the posted videos.
There is often a need to process the posted videos in order to support the various functions of the sharing websites. This processing can involve resizing the videos, compressing the videos, detecting objects within the videos, detecting images within the videos, etc. Certain types of processing can benefit from treating different regions of a video differently. For example, a resizing process can resize only certain regions of the video while leaving other regions at their native sizes. Likewise, a compression process can allocate a greater amount of data to important regions while providing fewer data to less important regions.
Thus, websites seek ways to segment videos into different regions in order to provide the ability to apply different processing to the different regions. Generally, the video should be segmented into regions of similar appearance. For example, black bars surrounding a letterboxed video can form a region, and objects within the video, such as grass, sky, or hair can also form separate regions. While image segmentation techniques exist, such techniques tend to perform poorly when applied in the video domain due to the lack of temporal coherence. Therefore, there is a need for a way to perform video segmentation.